Image processing apparatus and method for removing a facial object

ABSTRACT

An apparatus having an image acquiring unit configured to acquire the face image. The apparatus additionally having a first determining unit configured to determine a representative pixel value representing a skin region, based on pixels included in the face image, a region generating unit configured to generate a first region including the object, and a second determining unit configured to determine correction pixels, which are one or more pixels used for the correction, out of pixels present in a vicinity of the first region, based on similarity to the representative pixel value. The apparatus also having a partial-image generating unit configured to generate a partial image, which is a collection of pixels for replacing pixels corresponding to the object, using the correction pixels, and a correcting unit configured to perform correction by combining the partial image with the face image.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on Japanese Patent Application No. 2015-049733filed with the Japan Patent Office on Mar. 12, 2015, the entire contentsof which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an image processing technique fordeleting an unnecessary object included in an image.

Description of the Related Art

In recent years, a technique for automatically correcting an imageincluding a face of a person (a face image) photographed using a digitalcamera or the like is becoming more common. Deleting an unnecessaryobject included in an image is an example of such a technique. Forexample, by removing moles or wrinkles on a face, it is possible to makeskin look beautiful. By detecting and removing eyebrows, it is possibleto redraw the eyebrows.

As the technique for deleting an unnecessary object included in animage, for example, there is an image processing apparatus described inJapanese Patent Application Laid-Open No. 2011-170838. The imageprocessing apparatus searches for, in an image, a region similar to aregion where an unnecessary object is present and replaces theunnecessary object together with the region to thereby remove theunnecessary object.

SUMMARY OF THE INVENTION

If it is attempted to remove an unnecessary object (a mole, a wrinkle,an eyebrow, etc.) included in a face of a person using the apparatusdescribed in Japanese Patent Application Laid-Open No. 2011-170838,artificiality sometimes remains in an image after processing. This isbecause a shade caused by a way of irradiation of illumination or thelike is different for each part. For example, even if calculated values(e.g., totals of differences of luminance values) of pixels are similarbetween regions, if regions having different shades are selected, anunnatural correction result is obtained.

In order to solve this problem, a technique for removing only a removaltarget object without substantially changing pixel values around theremoval target object is necessary.

The present invention has been devised in view of the above problems andit is an object of the present invention to provide a technique forremoving an unnecessary object included in an image without a sense ofdiscomfort.

The present invention in its one aspect provides an image processingapparatus that performs correction for removing an object from a faceimage, which is an image including a face of a person, the imageprocessing apparatus comprises an image acquiring unit configured toacquire the face image; a representative-pixel-value determining unitconfigured to determine a representative pixel value, which is a pixelvalue representing a skin region, on the basis of pixels included in theface image; a region generating unit configured to generate a firstregion, which is a region including the object; a correction-pixeldetermining unit configured to determine correction pixels, which areone or more pixels used for the correction, out of a plurality of pixelspresent in a vicinity of the first region, on the basis of similarity tothe representative pixel value; a partial-image generating unitconfigured to generate a partial image, which is a collection of pixelsfor replacing pixels corresponding to the object, using the correctionpixels; and a correcting unit configured to perform correction bycombining the partial image with the face image.

The image processing apparatus according to the present invention is anapparatus that performs image correction for removing a target objectincluded in a face image. The target object may be anything such as aneyebrow, a mole, a wrinkle, a freckle, a birthmark, or a scar on thesurface of skin.

The representative-pixel-value determining unit is a unit thatdetermines a pixel value representing a surface region (a skin region)of a face included in an image. The representative pixel value can becalculated on the basis of, for example, a result obtained by sampling aplurality of pixels corresponding to skin.

The region generating unit is a unit that generates a region (a firstregion) including a removal target object. For example, when there isinformation concerning a feature point corresponding to the removaltarget object, the first region may be generated on the basis of theinformation or a user of the apparatus may directly designate the firstregion.

The correction-pixel determining unit is a unit that determines pixels(correction pixels) used in correcting the first region, that is,performing image processing for removing an object. The correctionpixels are selected on the basis of similarity to the representativepixel value out of pixels present in the vicinity of the first region.For example, the similarity may be determined on the basis of adifference in a luminance component from the representative pixel valueor may be determined on the basis of a difference in a hue.

The partial-image generating unit is a unit that generates, on the basisof the determined correction pixels, a partial pixel from which theobject is removed. The partial-image generating unit generates, forexample, on the basis of the correction pixels, as the partial image, animage obtained by interpolating the first region.

The partial image generated in this way is an image generated making useof information such as luminance and a hue of the pixels present aroundthe first region and on the basis of pixels having a pixel value closerto the representative pixel value. Therefore, a pixel value of thepartial image does not greatly fluctuate before and after correction.That is, it is possible to perform natural correction.

The correction pixels may include a plurality of pairs of firstcorrection pixels and second correction pixels opposed to each otheracross the first region. The partial-image generating unit may generate,for each of the pairs, a pixel row for connecting the first correctionpixel and the second correction pixel to generate a partial image inwhich the first region is interpolated. The color of the pixels includedin the pixel row may change stepwise from a color of the firstcorrection pixel to a color of the second correction pixel.

The correction pixels are desirably a pair of pixels opposed to eachother across the first region. Consequently, it is possible tointerpolate a color of the pixels present in the middle (the pixelspresent in the first region). By changing a color of the pixelsstepwise, it is possible to perform interpolation without a sense ofdiscomfort.

The first region may be a region having an elliptical shape. The firstcorrection pixels and the second correction pixels may be pixels opposedto each other across the first region along a minor axis direction ofthe elliptical shape.

When the first region has the elliptical shape, it is desirable toperform the interpolation along the minor axis direction. Consequently,an amount of information that should be interpolated decreases. It ispossible to obtain a more natural correction result.

The region generating unit may further generate a second region which isincluded in the first region, includes the object, and includes a skinregion smaller than the skin region of the first region. The correctingunit may correct the face image by combining the partial image insidethe second region.

The region for obtaining the correction pixels is desirably a regionincluding the skin region. On the other hand, the region to be correctedis desirably a small region including only the object. Therefore, aftergenerating the second region included in the first region, correctionmay be applied to the second region.

The partial-image generating unit may further apply a smoothing filterto the generated partial image.

When the pixel row for coupling the first correction pixels and thesecond correction pixels is simply generated, an edge sometimes occursalong a direction in which the pixels are generated. Therefore, byfurther applying the filter for smoothing the partial image, it ispossible to reduce the edge.

Note that the present invention can be specified as an image processingapparatus including at least a part of the units. The present inventioncan also be specified as an image processing method performed by theimage processing apparatus. The present invention can also be specifiedas a computer program for causing a computer to execute the imageprocessing method. The processing and the units can be freely combinedand implemented as long as technical contradiction does not occur.

According to the present invention, it is possible to remove anunnecessary object included in an image without a sense of discomfort.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system configuration diagram of an image processingapparatus according to an embodiment;

FIG. 2 is a flowchart of processing performed by the image processingapparatus;

FIG. 3 is a flowchart for explaining, in detail, processing foracquiring a representative pixel value;

FIG. 4 is a flowchart of processing for extracting a removal targetobject;

FIGS. 5A and 5B are examples of a face image and an eyebrow image;

FIG. 6 is a flowchart for explaining processing for generating acorrection region;

FIG. 7 is a diagram for explaining arrangement of feature pointscorresponding to an eyebrow;

FIG. 8 is a diagram showing a first eyebrow region determined on thebasis of the positions of the feature points;

FIG. 9 is a diagram showing a relation between the first eyebrow regionand a second eyebrow region; and

FIG. 10 is a diagram for explaining processing for generating a partialimage.

DESCRIPTION OF THE EMBODIMENTS

A preferred embodiment of the present invention is explained below withreference to the drawings.

An image processing apparatus according to this embodiment is anapparatus that acquires an image including a face and applies correctionfor removing an eyebrow to the image. FIG. 1 is a system configurationdiagram of an image processing apparatus 10 according to thisembodiment.

First Embodiment

The image processing apparatus 10 according to a first embodimentincludes an image acquiring unit 11, a representative-pixel-valueacquiring unit 12, an object extracting unit 13, a correction-regiongenerating unit 14, a partial-image generating unit 15, and an imagecorrecting unit 16.

The image acquiring unit 11 is a unit that acquires a processing targetimage (an image including a face of a person; hereinafter, face image).Typically, the image acquiring unit 11 is a storage device such as afixed disk drive or a flash memory. Note that the image acquiring unit11 may be a unit (e.g., an interface device or a radio communicationdevice) that acquires an image from the outside of the apparatus or maybe a unit that picks up an image via a lens and an image pickup element.The processing target image may be an image corresponding to framesforming a moving image.

Note that, in this embodiment, a color space of the face image is YCbCr.However, the color space may be RGB, Lab, or the like.

The representative-pixel-value acquiring unit 12 is a unit that acquiresa pixel value representing skin of a person included in the face image(hereinafter, representative pixel value). Specifically, therepresentative-pixel-value acquiring unit 12 samples a plurality ofpixels from a region corresponding to the surface of the face andcalculates a single pixel value on the basis of a sampling result. Aspecific processing method is explained below.

The object extracting unit 13 is a unit that detects an object thatshould be removed from the face image and segments an image includingthe object. Specifically, the object extracting unit 13 detects featurepoints corresponding to an eye and an eyebrow of a person from the faceimage and, after specifying a region where the eyebrow is present, clipsthe region. The feature points to be detected are desirably pointscorresponding to the contour of the eyebrow, points corresponding to theouter corner of the eye and the inner corner of the eye, and the like.In the following explanation, an image after the clipping is referred toas eyebrow image.

Note that, in an example explained in this embodiment, the removaltarget object is the eyebrow. However, the target object may be a mole,a wrinkle, or the like present on the surface of the face.

The correction-region generating unit 14 is a unit that generates atarget region where image processing for removing the eyebrow isperformed (hereinafter, first eyebrow region) and a region larger thanthe first eyebrow region and including the first eyebrow region(hereinafter, second eyebrow region). Details concerning the firsteyebrow region and the second eyebrow region and a specific processingmethod are explained below.

The partial-image generating unit 15 is a unit that generates a partialimage on the basis of information concerning the including regiongenerated by the correction-region generating unit 14 and therepresentative pixel value acquired by the representative-pixel-valueacquiring unit 12. The partial image is an image for performing removalof the eyebrow by being combined with the first eyebrow region.

The image correcting unit 16 is a unit that performs correction for theface image using the partial image generated by the partial-imagegenerating unit 15. When the image correcting unit 16 performs thecorrection, an unnecessary object (in this embodiment, the eyebrow)included in the face image is removed.

Note that the units explained above may be realized by exclusivelydesigned hardware or may be realized by software modules. The units maybe realized by a field programmable gate array (FPGA), an applicationspecific integrated circuit (ASIC), or the like or may be realized by acombination of these.

When the units are configured as software, a computer program stored inan auxiliary storage device is loaded to a main storage device andexecuted by a CPU, whereby the units function. (None of the CPU, theauxiliary storage device, and the main storage device is shown in thefigure.)

A unit that executes the computer program is not limited to the CPU andmay be an image signal processor (ISP), a digital signal processor(DSP), a graphics processing unit (GPU), or the like.

Processing for removing an eyebrow included in an acquired face image isexplained in order with reference to FIG. 2, which is a flowchart ofprocessing performed by the image processing apparatus 10. Theprocessing shown in FIG. 2 is started on the basis of an instructionfrom a user of the apparatus.

First, the image acquiring unit 11 acquires a face image (step S11). Inthis embodiment, a face image stored in advance is acquired from astorage device. However, a face image may be acquired via acommunication unit or an image pickup unit.

Subsequently, the representative-pixel-value acquiring unit 12 acquiresthe face image acquired by the image acquiring unit 11 and arepresentative pixel value, which is a pixel value representing a face(step S12). FIG. 3 is a flowchart for explaining, in detail, content ofthe processing performed in step S12.

First, the representative-pixel-value acquiring unit acquires a centerpoint of the face (step S12). The center point of the face can be set,for example, at the tip of a nose or in the vicinity of the nose.However, the center point is desirably set in a place less easilyaffected by a make-up and illumination.

Subsequently, the representative-pixel-value acquiring unit 12 arrangespoints for performing sampling of pixels (hereinafter, sampling points)in the vicinity of the detected center point (step S122). For example,the representative-pixel-value acquiring unit 12 arranges 256 samplingpoints in a region of 16×16 pixels centering on the center pointacquired in step S121. Note that the sampling points are arranged tooverlap pixels.

Subsequently, the representative-pixel-value acquiring unit 12 acquiresY components, Cb components, and Cr components for the respective pixelson which the sampling points are arranged and generates histograms ofvalues of the respective components (step S123).

Subsequently, the representative-pixel-value acquiring unit 12calculates averages μ for the generated three histograms, extracts onlypixels in which all of the three values belong to a predetermined rangecentering on the averages, and sets the pixels as sampling pixels (stepS124). For example, when a standard deviation is represented as σ andthe predetermined range is represented as ±2σ, only sampling pixels inwhich all of the Y components, the Cb components, and the Cr componentsare present in the range of ±2σ centering on the averages are extracted.

Subsequently, in step S125, the representative-pixel-value acquiringunit 12 determines a representative pixel value of a skin region usingthe extracted sampling pixels. The representative pixel value can bedetermined using, for example, the averages of the Y components, the Cbcomponents, and the Cr components of the sampling pixels. However, therepresentative pixel value may be determined using other methods.

The representative pixel value corresponding to the face is determinedby the processing explained above.

Subsequently, the object extracting unit 13 extracts a removal targeteyebrow from the face image and clips a region including the eyebrow togenerate an eyebrow image (step S13). FIG. 4 is a flowchart forexplaining, in detail, content of the processing performed in step S13.

First, in step S131, the object extracting unit 13 extracts arectangular region corresponding to the face included in the image(hereinafter, face region). A method of extracting a face region from animage is publicly known. Therefore, detailed explanation of the methodis omitted.

Subsequently, in step S132, the object extracting unit 13 detects theeyebrow included in the face region. In this embodiment, the objectextracting unit 13 detects, with a known method, feature pointscorresponding to the contour of the eyebrow and feature pointscorresponding to the outer corner of the eye and the inner corner of theeye. Note that, when failing in the detection of the feature points,after performing filtering, the object extracting unit 13 may detect theeyebrow through matching processing.

Subsequently, in step S133, the object extracting unit 13 performsnormalization of the image. Specifically, the object extracting unit 13clips the face region and generates a rectangular image having apredetermined size such that feature points corresponding to the leftend and the right end of the eyebrow are placed in predeterminedcoordinates. In generating the rectangular image, the object extractingunit 13 may perform conversion of resolution using any interpolationmethod. When the eyebrow included in the face image is not horizontal,the object extracting unit 13 may perform the clipping after performingprocessing for rotating the eyebrow to be horizontal using affinetransformation or the like.

Note that the eyebrow image may be a color image or may be a set of aplurality of images in which color information is separated. Forexample, when a color space of the face image is YCbCr, a set of threeimages in which Y components, Cb components, and Cr components arerespectively separated may be the eyebrow image.

FIG. 5A is an example of the face image. FIG. 5B is an example of theeyebrow image. Black points shown in FIG. 5B are feature pointsrepresenting the left end and the right end of the eyebrow. Note that,in this example, only the left eyebrow is set as a processing target.However, when both the eyebrows are set as processing targets, theprocessing explained in this specification only has to be repeated instep S133 and subsequent steps.

Subsequently, the correction-region generating unit generates acorrection region on the basis of the eyebrow image generated in stepS13 (step S14). FIG. 6 is a flowchart for explaining, in detail, contentof the processing performed in step S14.

First, in step S141, the correction-region generating unit 14 generatesa first eyebrow region. The first eyebrow region is a region where theshape of the eyebrow is represented by three ellipses.

The generation of the eyebrow region is performed using the featurepoints detected in step S132. FIG. 7 is a diagram showing a positionalrelation among feature points present around the eyebrow. Referencenumeral 701 denotes a contour line of the eyebrow. Black points in thefigure represent feature points. Among the feature points shown in FIG.7, an “eye outer corner point” and an “eye inner corner point” arefeature points detected with reference to the eye. The other points arefeature points detected with reference to the eyebrow (feature pointscorresponding to the contour of the eyebrow).

The first eyebrow region is formed by three ellipses, i.e., a leftcircle (reference numeral 801), a center circle (reference numeral 802),and a right circle (reference numeral 803) indicated by dotted lines inFIG. 8.

Note that, in this embodiment, the first eyebrow region is generatedusing the three ellipses. However, the first eyebrow region may have ashape other than the illustrated shape as long as the first eyebrowregion is a region including the eyebrow. The first eyebrow region maybe generated by a method other than the illustrated method.

The ellipses can be defined by X coordinates and Y coordinates of thecenters, vectors of major axes, and vectors of minor axes. A method ofdefining the respective ellipses using feature points is explainedbelow.

(1) Definition Method for the Left Circle (Reference Numeral 801)

Center X coordinate: an X coordinate of an intermediate point between aleft point and an upper left point

Center Y coordinate: a Y coordinate of an intermediate point between alower left point and the upper left point

Major axis vector: a vector connecting an upper point and a point (an Xcoordinate of the left point and a Y coordinate of an intermediate pointbetween the left point and the eye outer corner point)

Minor axis vector: A vector orthogonal to the major axis vector of theleft circle and having length equal to the distance between the upperpoint and the lower point

(2) Definition Method for the Center Circle (Reference Numeral 802)

Center X coordinate: an X coordinate of an intermediate point between acenter point of the left circle and a center point of the right circle

Center Y coordinate: a Y coordinate of the intermediate point betweenthe lower left point and the upper left point

Major axis vector: a vector connecting a right point and the left point

Minor axis vector: A vector orthogonal to the major axis vector of thecenter circle and having length equal to the distance between the upperpoint and the lower point

(3) Definition Method for the Right Circle (Reference Numeral 803)

Center X coordinate: an X coordinate of an intermediate point betweenthe right point and an upper right point

Center Y coordinate: a Y coordinate of an intermediate point between alower right point and the upper right point

Major axis vector: a vector connecting the upper point and a point (an Xcoordinate of the right point and a Y coordinate of an intermediatepoint between the right point and the eye inner corner point)

Minor axis vector: A vector orthogonal to the major axis vector of theright circle and having length equal to the distance between the upperpoint and the lower point

The first eyebrow region is generated by the method defined above.

Subsequently, the correction-region generating unit generates an ellipseregion (reference numeral 901) in which the first eyebrow region isinscribed and sets the ellipse region as a second eyebrow region (stepS142). FIG. 9 is a diagram showing a positional relation between thefirst eyebrow region and the second eyebrow region. Note that, in thisexample, the second eyebrow region is generated such that the firsteyebrow region is inscribed in the second eyebrow region. However, amethod of generating the second eyebrow region is not limited to thismethod as long as a region including the first eyebrow region andincluding a larger number of skin regions can be generated.

Subsequently, the partial-image generating unit 15 generates a partialimage for correcting the face image using the representative pixel valuegenerated in step S12 and the first eyebrow region and the secondeyebrow region generated in step S14 (step S15).

Before explaining the processing in step S15, an overview of a method inwhich the image processing apparatus according to this embodimentcorrects the face image is explained.

The first eyebrow region and the second eyebrow region are generated bythe processing up to step S14. The region where the eyebrow is presentis the first eyebrow region. Therefore, basically, the eyebrow can beremoved by arranging pixels of a skin color (e.g., pixels having therepresentative pixel value) on the inside of the first eyebrow region.However, if the inside of the first eyebrow region is painted by asingle pixel value, the image becomes unnatural because only the portionhas no shade.

Therefore, rather than directly combining the representative pixelvalue, the image processing apparatus according to this embodimentselects, on the basis of the representative pixel value, pixels used forcorrection (correction pixels) from the periphery of the eyebrow andcalculates, on the basis of the correction pixels, a pixel value inperforming the combination.

Consequently, it is possible to make use of information concerning ashade present around the eyebrow. Therefore, it is possible to obtain anatural correction result.

The processing content in step S15 is specifically explained withreference to FIG. 10. A region indicated by a dotted line in FIG. 10 isthe first eyebrow region. A region indicated by hatching is the secondeyebrow region.

In this embodiment, scanning is performed by a predetermined distancetoward the outer side from pixels present on the boundary line of thesecond eyebrow region to specify a pixel having a luminance componentclosest to a luminance component of the representative pixel value. Notethat similarity of luminances can be determined by, for example, anabsolute value of a difference between the luminances.

In an example shown in FIG. 10, it is assumed that, as a result ofperforming scanning toward a Y-axis positive direction from a pixel1001, a pixel having most similar luminance is found in a pixel 1003.Similarly, it is assumed that, as a result of performing scanning towarda Y-axis negative direction from the pixel 1002, a pixel having mostsimilar luminance is found in a pixel 1004.

A hue of the pixel 1003 is represented as A and a hue of the pixel 1004is represented as B.

Subsequently, a pixel row connecting the pixel 1003 and the pixel 1004is generated. When the pixel row is generated, linear interpolation isperformed to determine hues of pixels such that colors sequentiallytransition from the hue A to the hue B. Consequently, a pixel row isgenerated in which the hues transition stepwise with gradations from thepixel 1003 to the pixel 1004.

When this processing is performed for all X coordinates, an image inwhich the first eyebrow region is painted out is obtained. Finally, asmoothing filter is applied to the image. The image may be blurred by,for example, 3×3 blur filter. Consequently, it is possible to reduce anedge in the longitudinal direction.

An image obtained in this way is the partial image in the presentinvention.

The generated partial image is transmitted to the image correcting unit16. The image correcting unit 16 executes, using the acquired partialimage, processing for removing the eyebrow included in the face image(step S16).

Specifically, the image correcting unit 16 replaces the pixels includedin the second eyebrow region of the face image with pixels included inthe partial image. Note that, when the face image is separated into aplurality of channels of a Y component, a Cb component, a Cr component,and the like, after applying the processing only to the channels, theimage correcting unit 16 only has to integrate the plurality ofchannels.

A corrected image is stored in a not-shown storage device or transmittedto the outside and provided to the user.

Note that, in the correction of the image, only the pixels included inthe second eyebrow region may be simply replaced. However, in this case,the boundary of the second eyebrow region sometimes becomes unnatural.Therefore, a mixing ratio of the pixels included in the second eyebrowregion and the pixels included in the partial image may be graduallychanged near the boundary of the second eyebrow region. For example, amask image for gradually reducing a degree of the correction may begenerated further on the outer side than the boundary of the secondeyebrow region. The combination may be performed using the mask image.

As explained above, the image processing apparatus according to thisembodiment interpolates the inside of the first eyebrow region using thepixels present in the vicinity of the second eyebrow region to removethe eyebrow. The pixels used for the interpolation are pixels havingpixel values close to the representative pixel value of the face andhaving information concerning the shade around the eyebrow. Therefore,compared with when the interpolation is performed with a single color orwhen the pixels in the vicinity are simply copied, it is possible toperform natural correction.

Modifications

The embodiment explained above is only an example. The present inventioncan be changed and carried out as appropriate without departing from thespirit of the present invention.

For example, in the explanation of the embodiment, the example isexplained in which the eyebrow of the person is removed. However, atarget of the removal may be anything included in the face of theperson. The target of removal may be, for example, a mole, a freckle, alaugh line, a wrinkle, a birthmark, a dark ring around an eye, a scar, atreatment mark, a beard, or the like. In particular, as a target objectis longer, more natural correction can be performed.

Note that, when an object other than the eyebrow is set as the removaltarget, in step S141, a first region including the target object onlyhas to be generated and, in step S142, a second region including thefirst region and including a larger number of skin regions only has tobe generated.

In the explanation of the embodiment, the removal target is determinedon the basis of the detected feature points. However, the user of theapparatus may designate the removal target. For example, the user maydesignate a target region using a pointing device or the like.

In the explanation of the embodiment, as shown in FIG. 10, the pixel rowis generated in a direction along the Y axis. However, the pixel row maybe generated in a direction parallel to the minor axis of the ellipse(the first eyebrow region). As a direction in which the pixel row isgenerated is closer to the minor axis direction of the ellipse, thenumber of pixels to be interpolated further decreases. Therefore, morenatural correction result can be obtained.

In the explanation of the embodiment, the image processing apparatusthat applies the processing to the image stored in the storage device isexplained as the example. However, the image does not always have to beacquired from the storage device and, for example, may be acquired fromthe outside of the apparatus via a wired or wireless network.

The present invention may be carried out as an image pickup apparatushaving a function of removing any object by combining an image pickupunit that picks up an image and the image processing apparatus.

After the correction for removing the eyebrow is performed, correctionfor combining (drawing) an eyebrow anew may be performed.

What is claimed is:
 1. An image processing apparatus that performscorrection for removing an object from a face image, which is an imageincluding a face of a person, the image processing apparatus comprising:an image acquiring circuitry that acquires the face image; arepresentative-pixel-value determining circuitry that determines arepresentative pixel value, which is a pixel value representing a skinregion, on the basis of pixels included in the face image; a regiongenerating circuitry that generates a first region, which is a regionincluding the object; a correction-pixel determining circuitry thatdetermines correction pixels, which are one or more pixels used for thecorrection, out of a plurality of pixels present in a vicinity of thefirst region, on the basis of similarity to the representative pixelvalue, wherein the correction pixels include a plurality of pairs of afirst correction pixel and second correction pixel opposed to each otheracross the first region, each of the first and second correction pixelbeing specified by performing scanning a predetermined distance towardan outer side from a pixel present on a boundary line of the firstregion and specifying a pixel having a luminance component closest to aluminance component of the representative pixel value; a partial-imagegenerating circuitry that generates a partial image, which is acollection of pixels for replacing pixels corresponding to the object,using the correction pixels; and a correcting circuitry that performscorrection by combining the partial image with the face image, whereinthe partial-image generating circuitry generates, for each of the pairs,a pixel row for connecting the first correction pixel and the secondcorrection pixel to generate a partial image in which the first regionis interpolated, and wherein the color of the pixels included in thepixel row changes stepwise from a color of the first correction pixel toa color of the second correction pixel.
 2. The image processingapparatus according to claim 1, wherein the first region is a regionhaving an elliptical shape, and the first correction pixels and thesecond correction pixels are pixels opposed to each other across thefirst region along a minor axis direction of the elliptical shape. 3.The image processing apparatus according to claim 1, wherein the regiongenerating circuitry further generates a second region which is includedin the first region, includes the object, and includes a skin regionsmaller than the skin region of the first region, and the correctingcircuitry corrects the face image by combining the partial image insidethe second region.
 4. The image processing apparatus according to claim1, wherein the partial-image generating circuitry further applies asmoothing filter to the generated partial image.
 5. An image processingmethod performed by an image processing apparatus that performscorrection for removing an object from a face image, which is an imageincluding a face of a person, the image processing method comprising thesteps of: acquiring the face image; determining a representative pixelvalue, which is a pixel value representing a skin region, on the basisof pixels included in the face image; generating a first region, whichis a region including the object; determining correction pixels, whichare one or more pixels used for the correction, out of a plurality ofpixels present in a vicinity of the first region, on the basis ofsimilarity to the representative pixel value, wherein the correctionpixels include a plurality of pairs of a first correction pixel andsecond correction pixel opposed to each other across the first region,each of the first and second correction pixel being specified byperforming scanning a predetermined distance toward an outer side from apixel present on a boundary line of the first region and specifying apixel having a luminance component closest to a luminance component ofthe representative pixel value; generating a partial image, which is acollection of pixels for replacing pixels corresponding to the object,using the correction pixels; and performing correction by combining thepartial image with the face image, wherein the generating of the partialimage further includes generating, for each of the pairs, a pixel rowfor connecting the first correction pixel and the second correctionpixel to generate a partial image in which the first region isinterpolated, and wherein the color of the pixels included in the pixelrow changes stepwise from a color of the first correction pixel to acolor of the second correction pixel.
 6. A non-transitory computerreadable storing medium recording a computer program for causing acomputer to execute each of the steps of the image processing methodaccording to claim 5.